Add inner_dtypes to NestedDtype for sub-column dtype casting #230
Conversation
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #230 +/- ##
==========================================
+ Coverage 98.24% 98.93% +0.68%
==========================================
Files 14 14
Lines 1254 1315 +61
==========================================
+ Hits 1232 1301 +69
+ Misses 22 14 -8 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Click here to view all benchmarks. |
There was a problem hiding this comment.
Copilot reviewed 9 out of 9 changed files in this pull request and generated no comments.
Comments suppressed due to low confidence (1)
src/nested_pandas/series/ext_array.py:248
- The variable 'infered_inner_dtypes' appears to be misspelled. Consider renaming it to 'inferred_inner_dtypes' for clarity.
pa_array, infered_inner_dtypes = cls._box_pa_array(scalars, pa_type=pa_type)
There was a problem hiding this comment.
Copilot reviewed 9 out of 9 changed files in this pull request and generated no comments.
Comments suppressed due to low confidence (2)
src/nested_pandas/series/packer.py:246
- Ensure that merging inner_dtypes with the '|' operator handles duplicate keys as intended so that any overlapping field types are combined correctly without unintentionally overriding useful type information.
dtype = NestedDtype(inferred_dtype.pyarrow_dtype, inner_dtypes=inner_dtypes | inferred_dtype.inner_dtypes)
src/nested_pandas/series/ext_array.py:692
- When accumulating inner_dtypes from multiple items in _box_pa_array, verify that merging dtypes using update() does not lead to inconsistent or conflicting type information if the same field is encountered with different inferred types.
scalar, dtypes = cls._box_pa_scalar(v, pa_type=pa_type)
|
This approach is way too complicated, started a new implementation at #242 |
It is the first step for multiple nesting implementation.
This is a draft PR, because I found the implementation to be over-complicated. Probably the better solution is using list-struct as the default data representation instead of this
inner_dtypessystem.